Offline Bangla Handwriting Recognition with Sequential Detection of Characters/Diacritics

This presents an offline handwriting recognizer for Bangla script. In spite of being a major script, very little progress has been made in this field for Bangla. Here, we present a handwriting recognition unit with sequential detection of characters/diacritics. A faster R-CNN was used to spot the graphemes from word images and the results were merged to form a transcription. Transfer learning and data augmentation techniques were applied to increase the speed and accuracy of the process. We achieved a WER and CER of 21.5% and 8.9% respectively, which is the first reported transcription result for Bangla script

Design of an Offline Handwriting Recognition System Tested on the Bangla and Korean Scripts

This dissertation presents a flexible and robust offline handwriting recognition system which is tested on the Bangla and Korean scripts. Offline handwriting recognition is one of the most challenging and yet to be solved problems in machine learning. While a few popular scripts (like Latin) have received a lot of attention, many other widely used scripts (like Bangla) have seen very little progress. Features such as connectedness and vowels structured as diacritics make it a challenging script to recognize. A simple and robust design for offline recognition is presented which not only works reliably, but also can be used for almost any alphabetic writing system. The framework has been rigorously tested for Bangla and demonstrated how it can be transformed to apply to other scripts through experiments on the Korean script whose two-dimensional arrangement of characters makes it a challenge to recognize. The base of this design is a character spotting network which detects the location of different script elements (such as characters, diacritics) from an unsegmented word image. A transcript is formed from the detected classes based on their corresponding location information. This is the first reported lexicon-free offline recognition system for Bangla and achieves a Character Recognition Accuracy (CRA) of 94.8%. This is also one of the most flexible architectures ever presented. Recognition of Korean was achieved with a 91.2% CRA. Also, a powerful technique of autonomous tagging was developed which can drastically reduce the effort of preparing a dataset for any script. The combination of the character spotting method and the autonomous tagging brings the entire offline recognition problem very close to a singular solution. Additionally, a database named the Boise State Bangla Handwriting Dataset was developed. This is one of the richest offline datasets currently available for Bangla and this has been made publicly accessible to accelerate the research progress. Many other tools were developed and experiments were conducted to more rigorously validate this framework by evaluating the method against external datasets (CMATERdb 1.1.1, Indic Word Dataset and REID2019: Early Indian Printed Documents). Offline handwriting recognition is an extremely promising technology and the outcome of this research moves the field significantly ahead

Handwriting Recognition of Bangla and Similar Scripts

This research is about offline Bangla handwriting text recognition. Here we introduce a publicly accessible dataset, as well as a basic character recognition scheme. The dataset contains pages with a 104 word essay and a collection of 84 isolated alpha-numeric characters. All the components in the pages are tagged with the associated ground truth information. The character recognition scheme presented here uses zonal pixel counts, structural strokes and bag of features modeled with grid points using U-SURF descriptor as features. The maximum classification accuracy we obtain is 96.8% using an SVM classifier with a cubic kernel

Introducing the Boise State Bangla Handwriting Dataset and an Efficient Offline Recognizer of Isolated Bangla Characters

This introduces the Boise State Bangla Handwriting Dataset, a publicly accessible offline handwriting dataset of Bangla script. This can be found at https://scholarworks.boisestate.edu/saipl/1/ A basic character recognition method is presented where the features are extracted based on zonal pixel counts, structural strokes and grid points with U-SURF descriptors modeled with bag of features. Benchmarking with this approach on 3 other publicly available Bangla datasets is reported. The highest classification accuracy obtained with an SVM classifier based on a cubic kernel is 96.8%

Segmentation-Free Korean Handwriting Recognition Using Neural Network Training

The idea of segmentation-free handwriting recognition has been introduced within the rise of deep learning. This technique is designed to recognize any script language/symbols as long as feedable training image set exists. The VGG-16 convolutional neural network model is used as a character spotting network using Faster R-CNN. Through the process of manual tagging, the location, size, and types of recognizable symbols are provided to train the network. This approach has been tested previously on text written in the Bangla script, where it has shown over 90% of accuracy overall. For Bangla, the network is trained and tested on Boise State Bangla Handwriting dataset. For Korean, the network is trained using the PE_92 Handwritten Korean character image database and shows promising results

Digitization of Handwritten Chess Scoresheets with a BiLSTM Network

During an Over-the-Board (OTB) chess event, all players are required to record their moves strictly by hand, and later the event organizers are required to digitize these sheets for official records. This is a very time-consuming process, and in this paper we present an alternate workflow of digitizing scoresheets using a BiLSTM network. Starting with a pretrained network for standard Latin handwriting recognition, we imposed chess-specific restrictions and trained with our Handwritten Chess Scoresheet (HCS) dataset. We developed two post-processing strategies utilizing the facts that we have two copies of each scoresheet (both players are required to write the entire game), and we can easily check if a move is valid. The autonomous post-processing requires no human interaction and achieves a Move Recognition Accuracy (MRA) around 95%. The semi-autonomous approach, which requires requesting user input on unsettling cases, increases the MRA to around 99% while interrupting only on 4% moves. This is a major extension of the very first handwritten chess move recognition work reported by us in September 2021, and we believe this has the potential to revolutionize the scoresheet digitization process for the thousands of chess events that happen every day

Segmentation-Free Bangla Offline Handwriting Recognition Using Sequential Detection of Characters and Diacritics with a Faster R-CNN

This paper presents an offline handwriting recognition system for Bangla script using sequential detection of characters and diacritics with a Faster R-CNN. This is an entirely segmentation-free approach where the characters and associated diacritics are detected separately with different networks named C-Net and D-Net. Both of these networks were prepared with transfer learning from VGG-16. The essay scripts from the Boise State Bangla Handwriting Dataset along with standard data augmentation techniques were used for training and testing. The F1 scores for the C-Net and D-Net networks are 89.6% and 93.2% respectively. Afterwards, both of these detection modules were fused into a word recognition unit with CER (Character Error Rate) of 11.2% and WER (Word Error Rate) of 24.4%. A spell checker further minimized the errors to 8.9% and 21.5% respectively. This same method is likely to be equally effective on several other Abugida scripts similar to Bangla

Character Spotting and Autonomous Tagging: Offline Handwriting Recognition for Bangla, Korean and Other Alphabetic Scripts

This paper demonstrates a framework for offline handwriting recognition using character spotting and autonomous tagging which works for any alphabetic script. Character spotting builds on the idea of object detection to find character elements in unsegmented word images. An autonomous tagging approach is introduced which automates the production of a character image training set by estimating character locations in a word based on typical character size. Although scripts can vary vividly from each other, our proposed approach provides a simple and powerful workflow for unconstrained offline recognition that should work for any alphabetic script with few adjustments. Here we demonstrate this approach with handwritten Bangla, obtaining a character recognition accuracy (CRA) of 94.8% and 91.12% with precision and autonomous tagging, respectively. Furthermore, we explained how character spotting and autonomous tagging can be implemented for other alphabetic scripts. We demonstrated that with handwritten Hangul/Korean obtaining a Jamo recognition accuracy (JRA) of 93.16% using a tiny fraction of the PE92 training set. The combination of character spotting and autonomous tagging takes away one of the biggest frustrations—data annotation by hand, and thus, we believe this has the potential to revolutionize the growth of offline recognition development

Performance Comparison of Scanner and Camera-Acquired Data for Bangla Offline Handwriting Recognition

This paper presents a comparison of offline handwriting recognition performance between two different image acquisition devices: scanner and cell-phone camera. Whereas a flat-bed scanner offers higher quality distortion free imaging, a cell-phone camera trumps on the convenience and ease of use. The aim of this research is to quantify how the extra quality obtained from a scanner impacts the offline handwriting recognition. This was evaluated with two classification framework: a segmentation-free offline Bangla handwriting transcription with sequential detection of characters/diacritics and a Bangla handwritten digit recognizer with an SVM classifier. The Boise State Bangla Handwriting dataset is used for the experiments. The highest recognition rate came when both training and testing on scanned images. The network trained with scanned images also exceeded the one trained with camera-acquired images in recognition of the camera acquired images, despite the mismatch in source

Boise State Bangla Handwriting Dataset

The BSU Bangla Dataset is an offline handwriting dataset of Bangla, one of the major scripts in the world. The fundamental objective of this dataset is to foster the offline Bangla handwriting text recognition related researches. The easy availability and simple structure of this dataset are believed to help the research community in developing and testing such recognizers. This dataset is an anonymous and voluntary contribution of many people and the acquisition is still going on. The development of a strong handwritten text recognizer will help to digitally store handwritten archived literature, documents and contribute in digital life automation in many ways such as digital character conversion, meaning translation, content-based image retrieval, keyword spotting, signboard translation, text-to-speech conversion, scene image analysis, postal sorting, etc